Skip to content

RFC 11: Core Concepts - #656

Draft
govekk wants to merge 6 commits into
ome:mainfrom
govekk:rfc11
Draft

govekk wants to merge 6 commits into
ome:mainfrom
govekk:rfc11

Conversation

@govekk

@govekk govekk commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Authors of RFC 11 banded together at the 2026 NGFF Zurich hackathon to draft a glossary and helpful visuals for core concepts necessary to understand the OME-Zarr spec.

The goal is to be a primer for concepts that are necessary to undersand the OME-Zarr v1 spec, but as that is still in development, there are expected to be changes to the terms in this RFC.

Todo (before review):

  • Write proposal section
  • Other RFC sections: stakeholders, implementation, drawbacks, future
  • Set up regular meetings of authors
  • Update with rest of visuals

@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

Automated Review URLs

@govekk
govekk marked this pull request as draft November 5, 2026 14:55
Comment thread rfc/11/images/types_of_tools.png Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for validator, did you meant ome-ngff-validator instead?
You should also align the box heights (the last one is bigger), and add the dotted line on the Reader/writer (or remove the dotted lines from the others)

Comment thread rfc/11/index.md
Comment on lines +107 to +113
### Chunk

The fundamental unit of data in a Zarr array. An array is divided into chunks along each dimension according to the chunk grid. The chunks are the smallest units that can be read independently.

### Shard

A storage object that contains one or more chunks. Sharding reduces the number of objects in a store by grouping chunks together, which may improve performance on file systems and object storage.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't encourage defining things this way, I really think it will confuse people. IMO it's much better to stick with the concepts in the zarr spec.

The zarr spec defines chunks, which are individual stored objects containing a subregion of an array.

The zarr spec does not define "shards". It defines a sharding codec, that partitions a chunk into subchunks.

Zarr-python happens to use the terms "chunks" and "shards" the way the selected text does; I was responsible for that API choice and I regret it, please don't perpetuate this.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what you could say is that sometimes chunks are colloquially called "shards" when an array uses the sharding codec

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait sharding isn't even an official zarr term? that makes it even funnier 😆 thank you davis for the comic relief at least

@d-v-b d-v-b Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The chunks are the smallest units that can be read independently.

while we're here... this is also not strictly true. if you store chunks uncompressed, you can read / write less than a full chunk, down to individual array elements, if your storage backend allows it

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants